ebook sql server dba training plan

38
 ©Brent Ozar Unlimited® 2015 Page 1 htp://BrentOzar.com/needs How to Develop Y our DBA Career SQL Server DBA Training Plan 3rd Edition, Spring 2015 - BrentOzar.com/needs

Upload: lexanx

Post on 07-Jul-2018

215 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 138

copyBrent Ozar Unlimitedreg 2015 Page 1 htpBrentOzarcomneeds

How to Develop Your DBA CareerSQL Server DBA

Training Plan3rd Edition Spring 2015 - BrentOzarcomneeds

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 238

copyBrent Ozar Unlimitedreg 2015 Page 2 htpBrentOzarcomneeds

Yoursquore a developer or a sysadmin who wants to become a DBAWe know Wersquove been there too and wersquore here to share our lessons learned

so that your journey is easier than ours Wersquore Brent Ozar Unlimitedreg aconsulting company that makes SQL Server faster and more reliable

I learned to be a DBA thehard way The hard crappyway Our SQL Server was introuble and I was the kindof person who would roll up

my sleeves and 1047297gure outwhatever was broken Nextthing you know I was theone responsible for it

And boy did that suck

I didnt know about any free videos or blogsor e-books I didnt have thebudget to go to a class andeven if I did I didnt know whereto 1047297nd a good one DBA wasnt

in my job title and it wouldntbe for years

We want to make your learningexperience much beter thanmine was

This book - part of our free 6-Month DBA Training Plan -covers the basics of the topicplus link to our favorite free

training material on that topic

Your journey will start withOzarrsquos Hierarchy of DatabaseNeeds the pyramid shown atright In the next six months

well take you from the botomof the pyramid up to the top

You may not be able to 1047297x everything in yourenvironment during those six months but atleast youll understand the work involvedand how to con1047297dently get started

Weve collected thousands of resourcesover the years -training videos scripts blogposts about everythingfrom indexing to careers - and were excitedto share them with you Its all about makingyour journey to Professional Database

Administrator easier than ours was

Welcome to Your Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 338

copyBrent Ozar Unlimitedreg 2015 Page 3 htpBrentOzarcomneeds

If You Have Questions

If you have questions about what yourereading start by Googling your questions It

sounds obvious but youd be amazed at howmuch good stuff there is out there to help(Im not being sarcastic This is exactly how

we get started whenever we have our ownquestions)

If you have a really short question and youexpect a really short answer and yoursquore onTwiter tweet the question with SQLhelp inyour tweet Lots of community membersmonitor this hash tag so even if they donrsquotfollow you personally theyrsquoll still see the

tweet and respond You can read more aboutSQLhelp too

If youd like to post a question tryDBAStackExchangecom or

SQLServerCentrals forums Yes both ofthese require registration but theyre totallyworth it On both of these sites thereshundreds - sometimes thousands - of peoplewho are itching to help answer your

questions They react fast too - make sureto go back and revisit your question every10-15 minutes for the 1047297rst few hours to seewhats happening Answer their clari1047297cation

questions and include as much detail as youcan For more instructions read Geting Help

with a Slow QueryIf you still cant get the answers you needemail us at HelpBrentOzarcom This is a

real email address manned by the realpeople at Brent Ozar Unlimited This isntone of those emails where it says Dont hitrespond because nobody cares Seriouslywe care and thats why we put these emailstogether Just please dont use that as your

FIRST resort - were real people with real jobs and real families and theres only somany hours per week that we can spendanswering questions By using the abovemethods 1047297rst youll be able to leverage thewhole communitys expertise instead of just

a few of us

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 438

copyBrent Ozar Unlimitedreg 2015 Page 4 htpBrentOzarcomneeds

Letrsquos start by making an Excel spreadsheet of the servers wersquove got the

number of databases on each one and over the coming weeks wersquoll 1047297x them

Build a Server Inventory

by Brent Ozar

At your company walk intothe VP of Saless officeand ask them how manysalespeople they have

NO I mean dont actuallyDO that because hes

going to ask you why the sales app is so

slow But I mean imagine if you COULD walkinto his office and ask him that I bet hewould have an instant answer He wouldntwait for a single moment Or walk into theCEOs office and ask how many employeeshe has Or ask the CFO how much the annualbudget is

My point is that when youre in charge youneed to know exactly what youre in chargeof

Make a Spreadsheet InventoryLets start by making a spreadsheet Acrossthe top make columns for

bull SQL Server Version (2012 2008 2005)

bull Edition (Standard Enterprise Developer)

bull Environment (Production QAdevelopment disaster recovery)

bull Department (sales HR accounting ITmixed use)

bull Business Users Aff ected (list of people toemail when the server dies)

bull Application Names (internal or externalproduct names)

bull Plan B

That last column gets a litle tricky - itmeans if this server dies in a 1047297re whats our

Plan B Are we going to restore thedatabases from another server Will we failover to a log shipped copy Or will weupdate our resume and head out for an earlylunch As we go farther into the trainingwere going to get much more speci1047297c about

Plan B

Theres no wrong answers here - week 1 isabout understanding where were at todaynot where wed like to be Were never where

wed like to be (Me personally Id like to beat a poolside bar right now but noooo Im ina hotel room waiting for my girlfriend to1047297nish blow drying her hair If youve everwondered why I write so much you can thankher full head of hair)

If youd like to get ambitious add additionalcolumns for Core Count CPU Count andMemory The core and CPU counts will getyou a head start on licensing although Ihave to confess that were not going to cover

licensing as part of our training plan

What Well Do With This Spreadsheet

Right now you probably sleep well at night

thinking you know everything thatshappening in these servers Hoooweee haveI got bad news for you Over the next sixmonths were going to progressively add

more and more columns to this spreadsheetas we learn more about our environment

uncover problems and learn how to solvethem

For bonus points add a column for WhatScares Me Write a quick note about the one

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 538

copyBrent Ozar Unlimitedreg 2015 Page 5 htpBrentOzarcomneeds

thing that scares youmost about this serverMaybe its blockingproblems maybe its the

failing jobs maybe itscode you dont

understand Six monthsfrom now I bet youll be proud of how thiscolumn has changed

How to Survey Your Network for Servers

Put a row in the spreadsheet for every serveryou have - whether youre in charge of it ornot We want to start with a good inventoryof what we have and theres two good free

tools to do it

Microsof Assessment and Planning Toolkit -

its actually designed for licensingcompliance but it works great for building

server inventories It scans your networklooking for whatever programs you pick but

just con1047297ne it to SQL Servers only

Quest Discovery Wizard for SQL Server - itsa GUI tool that pings all the servers in your

network and tries to 1047297gure out if theyve gotSQL Server installed If youre in a smallshop where your account has admin

privileges in the domain you might 1047297nd a lotmore servers than you expected

We dont get paid forplugging these productsand were always on thelookout for similar

inventory-building toolsso if you know of a beter

one email it to us atHelpBrentOzarcom

PSST Ask About This Before Geting Hired

When you take a new job as a DBA the very1047297rst question you should ask the company isDo you have a list handy of all the SQLServers Ill be managing I dont have to seethe list - I understand if you have security

concerns - but I just want to know if that listexists

Most of the timeit wont

This question serves two purposes it tellsYOU if the company has their act togetherwhen it comes to documentation and it tellsTHEM that youre the right person tomanage their database servers If they donthave the list theyre going to want that list

right away Nows your chance to explainhow you would go about gathering thatinformation (armed with the info in this

email)Bonus points create a SQL Server SupportMatrix a document that explains for yourdevelopers and end users whats allowed inproduction DR QA and development Thishelps set expectations going forward - if aservers going to be production then it has

to be stable and that means making surechanges dont happen accidentally

I created that sample support matrix when I

worked as a DBA and Ive shared it so youcan do a save-as and get a fast start on your

own Hope that helps

Most DBAs donrsquot

actually have a list of

their servers

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638

copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds

by Brent Ozar

The only reason we dobackups is so we can dorestores

When I 1047297rst started outas a SQL Server DBA Ithought things were

going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent

every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be

In theory you test our backup strategyahead of time with Kendras 5 Simple

Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers

In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its

just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a

database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier

Where to Do Restores

When youre restoring code (storedprocedures views triggers etc) or

individual tables dont restore onto the

production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a

diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments

Afer weve safely restored the right dataonto another server its easy to copy that

data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked

server tables

However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the

data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase

This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that

Bootcamps and

certi 1047297 cation

programs alone wonrsquot get

you that 1047297 rst jobmdash not when

your competition has

experience

(And yes this means wersquore assuming yoursquore backing them up)

The Reason We Back Up

All Those Databases

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738

copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds

particular server didnt have any otheractivity happening due to the outage Were

not always so lucky but it helps to think outof the box like that

A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that

youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase

Doing the Restore

Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to

apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore

When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or

transaction logbackups The goal is to

1047297nish the restorequickly

Next if diff erential

backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one

Next restore all of the transaction log

backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -

again using WITH NORECOVERYDoing all of this with the GUI sucks The

more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and

restores them for you automatically in orderWell talk about automating restores in the

next training module

To learn more about backups and restoresour favorite geting-started articles are

bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA

bull Brents DBA Nightmare SQL Down NoPlans

bull Jess 3 Things You Need to Start Doing toYour Database Server

When I Did My First Restore

I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and

early to print out the list of labels for the highschool graduates weddings babies and so

on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld

One morning as part of my

data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the

WHERE clause so Ideleted all of the photoshoots ever

Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the

restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it

Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested

Either the backups are

tested or yoursquore about

to be tested

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838

copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds

by Brent Ozar

If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way

too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction

logs)

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0100trn

bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn

bull MyDatabase_20130718_0400trn

bull MyDatabase_20130718_0500trn

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

In that scenario I took my full backup at

midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in

the real world but

its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)

If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les

live and presto the script will restore all ofthe 1047297les for you automatically

This Script Has Implications For You

You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and

see what backups have been done

It assumes the same databasepath when you restore If yourethe kind of DBA who likes to

change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and

The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves

As we script out a restore wersquoll learn how we need to design backups

Doing Faster Database Restores

Knowing how yoursquoll

restore aff ects how

you back up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938

copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds

development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le

names at restore time but theyre not quiteas elegant when it comes to restoring fulls

diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully

automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly

load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact

You can automate your own 1047297re drill testing

Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a

series of Agent jobs to restore a diff erentservers backups every day and then run

DBCC jobs on them This way I know Imgeting really good backups

Your Homework for This Chapter

You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way

Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or

staging environment If you get errors leavecomments on that blog post describing the

error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly

in a GUI - youll just open a proven script andhit Execute Bam

Geting Even Fancier

You can use this technique to build smallercopies of your production databases For

example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring

the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down

The end result was a database that was over50 smaller

I then backed it up with compression and

lef

those backups in a folder that thedevelopers could access That made it easier

for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired

Free 30-Minute Videos to Learn More

Log Shipping Part 1 Jes explains how logshipping is really just doing your restores

ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs

Backup Planning Video Jeremiah explains 3

company scenarios and asks you to pick theright backup strategy for each

Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about

backups We talk database mirroring

clustering AlwaysOn Availability Groupsand more

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038

copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds

by Brent Ozar

Run our free sp_Blitztradestored procedure on one of

your production servers andpay particular atention tothe Security section of theresults It lists the logins

who have been granted thesysadmin or security admin roles

Dont think of them as logins

Think of them as people who can get you1047297red

These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop

Youre probably not get

ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in

Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is

unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room

How to Get Started Locking Things Down

Before you start removing peoples SArights be aware that there can be political

backlash In one shop the head developers

SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in

it and then dropped the database Nobody

knew because it was only around for 30-45minutes The problem could have been

avoided by communicating the securitychanges ahead of time and thats where we

need to startTake the sp_Blitztrade output to your manager -

just the security portions - and say

something like this

Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont

want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that

Before we try to lock things down 1047297nd out who has copies of keys

Security Knowing Who Has

Access to the Servers amp Data

Think of sysadmin

logins as people

who can get you 1047297 red

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138

copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds

Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say

Were starting to lock down the productionserver but I want to make sure you have all

the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change

When You Get Resistance

When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a

large stake in the database For example ifthe accounting database is on the server goto the CFO and say

Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page

You want to turn it into a business problemnot a technology problem and the CFO will

very much be on your side She cant aff ordto have her entire department go down just

because some developer didnt put aWHERE clause on a T-SQL statement

I Know This Chapter Isnt Fun

Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks

In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details

Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those

applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want

Manage lots of

servers Learn

How to Be a

Senior DBA

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 2: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 238

copyBrent Ozar Unlimitedreg 2015 Page 2 htpBrentOzarcomneeds

Yoursquore a developer or a sysadmin who wants to become a DBAWe know Wersquove been there too and wersquore here to share our lessons learned

so that your journey is easier than ours Wersquore Brent Ozar Unlimitedreg aconsulting company that makes SQL Server faster and more reliable

I learned to be a DBA thehard way The hard crappyway Our SQL Server was introuble and I was the kindof person who would roll up

my sleeves and 1047297gure outwhatever was broken Nextthing you know I was theone responsible for it

And boy did that suck

I didnt know about any free videos or blogsor e-books I didnt have thebudget to go to a class andeven if I did I didnt know whereto 1047297nd a good one DBA wasnt

in my job title and it wouldntbe for years

We want to make your learningexperience much beter thanmine was

This book - part of our free 6-Month DBA Training Plan -covers the basics of the topicplus link to our favorite free

training material on that topic

Your journey will start withOzarrsquos Hierarchy of DatabaseNeeds the pyramid shown atright In the next six months

well take you from the botomof the pyramid up to the top

You may not be able to 1047297x everything in yourenvironment during those six months but atleast youll understand the work involvedand how to con1047297dently get started

Weve collected thousands of resourcesover the years -training videos scripts blogposts about everythingfrom indexing to careers - and were excitedto share them with you Its all about makingyour journey to Professional Database

Administrator easier than ours was

Welcome to Your Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 338

copyBrent Ozar Unlimitedreg 2015 Page 3 htpBrentOzarcomneeds

If You Have Questions

If you have questions about what yourereading start by Googling your questions It

sounds obvious but youd be amazed at howmuch good stuff there is out there to help(Im not being sarcastic This is exactly how

we get started whenever we have our ownquestions)

If you have a really short question and youexpect a really short answer and yoursquore onTwiter tweet the question with SQLhelp inyour tweet Lots of community membersmonitor this hash tag so even if they donrsquotfollow you personally theyrsquoll still see the

tweet and respond You can read more aboutSQLhelp too

If youd like to post a question tryDBAStackExchangecom or

SQLServerCentrals forums Yes both ofthese require registration but theyre totallyworth it On both of these sites thereshundreds - sometimes thousands - of peoplewho are itching to help answer your

questions They react fast too - make sureto go back and revisit your question every10-15 minutes for the 1047297rst few hours to seewhats happening Answer their clari1047297cation

questions and include as much detail as youcan For more instructions read Geting Help

with a Slow QueryIf you still cant get the answers you needemail us at HelpBrentOzarcom This is a

real email address manned by the realpeople at Brent Ozar Unlimited This isntone of those emails where it says Dont hitrespond because nobody cares Seriouslywe care and thats why we put these emailstogether Just please dont use that as your

FIRST resort - were real people with real jobs and real families and theres only somany hours per week that we can spendanswering questions By using the abovemethods 1047297rst youll be able to leverage thewhole communitys expertise instead of just

a few of us

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 438

copyBrent Ozar Unlimitedreg 2015 Page 4 htpBrentOzarcomneeds

Letrsquos start by making an Excel spreadsheet of the servers wersquove got the

number of databases on each one and over the coming weeks wersquoll 1047297x them

Build a Server Inventory

by Brent Ozar

At your company walk intothe VP of Saless officeand ask them how manysalespeople they have

NO I mean dont actuallyDO that because hes

going to ask you why the sales app is so

slow But I mean imagine if you COULD walkinto his office and ask him that I bet hewould have an instant answer He wouldntwait for a single moment Or walk into theCEOs office and ask how many employeeshe has Or ask the CFO how much the annualbudget is

My point is that when youre in charge youneed to know exactly what youre in chargeof

Make a Spreadsheet InventoryLets start by making a spreadsheet Acrossthe top make columns for

bull SQL Server Version (2012 2008 2005)

bull Edition (Standard Enterprise Developer)

bull Environment (Production QAdevelopment disaster recovery)

bull Department (sales HR accounting ITmixed use)

bull Business Users Aff ected (list of people toemail when the server dies)

bull Application Names (internal or externalproduct names)

bull Plan B

That last column gets a litle tricky - itmeans if this server dies in a 1047297re whats our

Plan B Are we going to restore thedatabases from another server Will we failover to a log shipped copy Or will weupdate our resume and head out for an earlylunch As we go farther into the trainingwere going to get much more speci1047297c about

Plan B

Theres no wrong answers here - week 1 isabout understanding where were at todaynot where wed like to be Were never where

wed like to be (Me personally Id like to beat a poolside bar right now but noooo Im ina hotel room waiting for my girlfriend to1047297nish blow drying her hair If youve everwondered why I write so much you can thankher full head of hair)

If youd like to get ambitious add additionalcolumns for Core Count CPU Count andMemory The core and CPU counts will getyou a head start on licensing although Ihave to confess that were not going to cover

licensing as part of our training plan

What Well Do With This Spreadsheet

Right now you probably sleep well at night

thinking you know everything thatshappening in these servers Hoooweee haveI got bad news for you Over the next sixmonths were going to progressively add

more and more columns to this spreadsheetas we learn more about our environment

uncover problems and learn how to solvethem

For bonus points add a column for WhatScares Me Write a quick note about the one

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 538

copyBrent Ozar Unlimitedreg 2015 Page 5 htpBrentOzarcomneeds

thing that scares youmost about this serverMaybe its blockingproblems maybe its the

failing jobs maybe itscode you dont

understand Six monthsfrom now I bet youll be proud of how thiscolumn has changed

How to Survey Your Network for Servers

Put a row in the spreadsheet for every serveryou have - whether youre in charge of it ornot We want to start with a good inventoryof what we have and theres two good free

tools to do it

Microsof Assessment and Planning Toolkit -

its actually designed for licensingcompliance but it works great for building

server inventories It scans your networklooking for whatever programs you pick but

just con1047297ne it to SQL Servers only

Quest Discovery Wizard for SQL Server - itsa GUI tool that pings all the servers in your

network and tries to 1047297gure out if theyve gotSQL Server installed If youre in a smallshop where your account has admin

privileges in the domain you might 1047297nd a lotmore servers than you expected

We dont get paid forplugging these productsand were always on thelookout for similar

inventory-building toolsso if you know of a beter

one email it to us atHelpBrentOzarcom

PSST Ask About This Before Geting Hired

When you take a new job as a DBA the very1047297rst question you should ask the company isDo you have a list handy of all the SQLServers Ill be managing I dont have to seethe list - I understand if you have security

concerns - but I just want to know if that listexists

Most of the timeit wont

This question serves two purposes it tellsYOU if the company has their act togetherwhen it comes to documentation and it tellsTHEM that youre the right person tomanage their database servers If they donthave the list theyre going to want that list

right away Nows your chance to explainhow you would go about gathering thatinformation (armed with the info in this

email)Bonus points create a SQL Server SupportMatrix a document that explains for yourdevelopers and end users whats allowed inproduction DR QA and development Thishelps set expectations going forward - if aservers going to be production then it has

to be stable and that means making surechanges dont happen accidentally

I created that sample support matrix when I

worked as a DBA and Ive shared it so youcan do a save-as and get a fast start on your

own Hope that helps

Most DBAs donrsquot

actually have a list of

their servers

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638

copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds

by Brent Ozar

The only reason we dobackups is so we can dorestores

When I 1047297rst started outas a SQL Server DBA Ithought things were

going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent

every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be

In theory you test our backup strategyahead of time with Kendras 5 Simple

Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers

In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its

just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a

database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier

Where to Do Restores

When youre restoring code (storedprocedures views triggers etc) or

individual tables dont restore onto the

production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a

diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments

Afer weve safely restored the right dataonto another server its easy to copy that

data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked

server tables

However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the

data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase

This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that

Bootcamps and

certi 1047297 cation

programs alone wonrsquot get

you that 1047297 rst jobmdash not when

your competition has

experience

(And yes this means wersquore assuming yoursquore backing them up)

The Reason We Back Up

All Those Databases

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738

copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds

particular server didnt have any otheractivity happening due to the outage Were

not always so lucky but it helps to think outof the box like that

A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that

youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase

Doing the Restore

Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to

apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore

When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or

transaction logbackups The goal is to

1047297nish the restorequickly

Next if diff erential

backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one

Next restore all of the transaction log

backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -

again using WITH NORECOVERYDoing all of this with the GUI sucks The

more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and

restores them for you automatically in orderWell talk about automating restores in the

next training module

To learn more about backups and restoresour favorite geting-started articles are

bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA

bull Brents DBA Nightmare SQL Down NoPlans

bull Jess 3 Things You Need to Start Doing toYour Database Server

When I Did My First Restore

I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and

early to print out the list of labels for the highschool graduates weddings babies and so

on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld

One morning as part of my

data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the

WHERE clause so Ideleted all of the photoshoots ever

Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the

restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it

Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested

Either the backups are

tested or yoursquore about

to be tested

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838

copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds

by Brent Ozar

If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way

too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction

logs)

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0100trn

bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn

bull MyDatabase_20130718_0400trn

bull MyDatabase_20130718_0500trn

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

In that scenario I took my full backup at

midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in

the real world but

its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)

If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les

live and presto the script will restore all ofthe 1047297les for you automatically

This Script Has Implications For You

You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and

see what backups have been done

It assumes the same databasepath when you restore If yourethe kind of DBA who likes to

change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and

The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves

As we script out a restore wersquoll learn how we need to design backups

Doing Faster Database Restores

Knowing how yoursquoll

restore aff ects how

you back up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938

copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds

development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le

names at restore time but theyre not quiteas elegant when it comes to restoring fulls

diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully

automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly

load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact

You can automate your own 1047297re drill testing

Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a

series of Agent jobs to restore a diff erentservers backups every day and then run

DBCC jobs on them This way I know Imgeting really good backups

Your Homework for This Chapter

You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way

Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or

staging environment If you get errors leavecomments on that blog post describing the

error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly

in a GUI - youll just open a proven script andhit Execute Bam

Geting Even Fancier

You can use this technique to build smallercopies of your production databases For

example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring

the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down

The end result was a database that was over50 smaller

I then backed it up with compression and

lef

those backups in a folder that thedevelopers could access That made it easier

for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired

Free 30-Minute Videos to Learn More

Log Shipping Part 1 Jes explains how logshipping is really just doing your restores

ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs

Backup Planning Video Jeremiah explains 3

company scenarios and asks you to pick theright backup strategy for each

Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about

backups We talk database mirroring

clustering AlwaysOn Availability Groupsand more

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038

copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds

by Brent Ozar

Run our free sp_Blitztradestored procedure on one of

your production servers andpay particular atention tothe Security section of theresults It lists the logins

who have been granted thesysadmin or security admin roles

Dont think of them as logins

Think of them as people who can get you1047297red

These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop

Youre probably not get

ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in

Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is

unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room

How to Get Started Locking Things Down

Before you start removing peoples SArights be aware that there can be political

backlash In one shop the head developers

SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in

it and then dropped the database Nobody

knew because it was only around for 30-45minutes The problem could have been

avoided by communicating the securitychanges ahead of time and thats where we

need to startTake the sp_Blitztrade output to your manager -

just the security portions - and say

something like this

Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont

want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that

Before we try to lock things down 1047297nd out who has copies of keys

Security Knowing Who Has

Access to the Servers amp Data

Think of sysadmin

logins as people

who can get you 1047297 red

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138

copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds

Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say

Were starting to lock down the productionserver but I want to make sure you have all

the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change

When You Get Resistance

When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a

large stake in the database For example ifthe accounting database is on the server goto the CFO and say

Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page

You want to turn it into a business problemnot a technology problem and the CFO will

very much be on your side She cant aff ordto have her entire department go down just

because some developer didnt put aWHERE clause on a T-SQL statement

I Know This Chapter Isnt Fun

Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks

In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details

Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those

applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want

Manage lots of

servers Learn

How to Be a

Senior DBA

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 3: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 338

copyBrent Ozar Unlimitedreg 2015 Page 3 htpBrentOzarcomneeds

If You Have Questions

If you have questions about what yourereading start by Googling your questions It

sounds obvious but youd be amazed at howmuch good stuff there is out there to help(Im not being sarcastic This is exactly how

we get started whenever we have our ownquestions)

If you have a really short question and youexpect a really short answer and yoursquore onTwiter tweet the question with SQLhelp inyour tweet Lots of community membersmonitor this hash tag so even if they donrsquotfollow you personally theyrsquoll still see the

tweet and respond You can read more aboutSQLhelp too

If youd like to post a question tryDBAStackExchangecom or

SQLServerCentrals forums Yes both ofthese require registration but theyre totallyworth it On both of these sites thereshundreds - sometimes thousands - of peoplewho are itching to help answer your

questions They react fast too - make sureto go back and revisit your question every10-15 minutes for the 1047297rst few hours to seewhats happening Answer their clari1047297cation

questions and include as much detail as youcan For more instructions read Geting Help

with a Slow QueryIf you still cant get the answers you needemail us at HelpBrentOzarcom This is a

real email address manned by the realpeople at Brent Ozar Unlimited This isntone of those emails where it says Dont hitrespond because nobody cares Seriouslywe care and thats why we put these emailstogether Just please dont use that as your

FIRST resort - were real people with real jobs and real families and theres only somany hours per week that we can spendanswering questions By using the abovemethods 1047297rst youll be able to leverage thewhole communitys expertise instead of just

a few of us

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 438

copyBrent Ozar Unlimitedreg 2015 Page 4 htpBrentOzarcomneeds

Letrsquos start by making an Excel spreadsheet of the servers wersquove got the

number of databases on each one and over the coming weeks wersquoll 1047297x them

Build a Server Inventory

by Brent Ozar

At your company walk intothe VP of Saless officeand ask them how manysalespeople they have

NO I mean dont actuallyDO that because hes

going to ask you why the sales app is so

slow But I mean imagine if you COULD walkinto his office and ask him that I bet hewould have an instant answer He wouldntwait for a single moment Or walk into theCEOs office and ask how many employeeshe has Or ask the CFO how much the annualbudget is

My point is that when youre in charge youneed to know exactly what youre in chargeof

Make a Spreadsheet InventoryLets start by making a spreadsheet Acrossthe top make columns for

bull SQL Server Version (2012 2008 2005)

bull Edition (Standard Enterprise Developer)

bull Environment (Production QAdevelopment disaster recovery)

bull Department (sales HR accounting ITmixed use)

bull Business Users Aff ected (list of people toemail when the server dies)

bull Application Names (internal or externalproduct names)

bull Plan B

That last column gets a litle tricky - itmeans if this server dies in a 1047297re whats our

Plan B Are we going to restore thedatabases from another server Will we failover to a log shipped copy Or will weupdate our resume and head out for an earlylunch As we go farther into the trainingwere going to get much more speci1047297c about

Plan B

Theres no wrong answers here - week 1 isabout understanding where were at todaynot where wed like to be Were never where

wed like to be (Me personally Id like to beat a poolside bar right now but noooo Im ina hotel room waiting for my girlfriend to1047297nish blow drying her hair If youve everwondered why I write so much you can thankher full head of hair)

If youd like to get ambitious add additionalcolumns for Core Count CPU Count andMemory The core and CPU counts will getyou a head start on licensing although Ihave to confess that were not going to cover

licensing as part of our training plan

What Well Do With This Spreadsheet

Right now you probably sleep well at night

thinking you know everything thatshappening in these servers Hoooweee haveI got bad news for you Over the next sixmonths were going to progressively add

more and more columns to this spreadsheetas we learn more about our environment

uncover problems and learn how to solvethem

For bonus points add a column for WhatScares Me Write a quick note about the one

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 538

copyBrent Ozar Unlimitedreg 2015 Page 5 htpBrentOzarcomneeds

thing that scares youmost about this serverMaybe its blockingproblems maybe its the

failing jobs maybe itscode you dont

understand Six monthsfrom now I bet youll be proud of how thiscolumn has changed

How to Survey Your Network for Servers

Put a row in the spreadsheet for every serveryou have - whether youre in charge of it ornot We want to start with a good inventoryof what we have and theres two good free

tools to do it

Microsof Assessment and Planning Toolkit -

its actually designed for licensingcompliance but it works great for building

server inventories It scans your networklooking for whatever programs you pick but

just con1047297ne it to SQL Servers only

Quest Discovery Wizard for SQL Server - itsa GUI tool that pings all the servers in your

network and tries to 1047297gure out if theyve gotSQL Server installed If youre in a smallshop where your account has admin

privileges in the domain you might 1047297nd a lotmore servers than you expected

We dont get paid forplugging these productsand were always on thelookout for similar

inventory-building toolsso if you know of a beter

one email it to us atHelpBrentOzarcom

PSST Ask About This Before Geting Hired

When you take a new job as a DBA the very1047297rst question you should ask the company isDo you have a list handy of all the SQLServers Ill be managing I dont have to seethe list - I understand if you have security

concerns - but I just want to know if that listexists

Most of the timeit wont

This question serves two purposes it tellsYOU if the company has their act togetherwhen it comes to documentation and it tellsTHEM that youre the right person tomanage their database servers If they donthave the list theyre going to want that list

right away Nows your chance to explainhow you would go about gathering thatinformation (armed with the info in this

email)Bonus points create a SQL Server SupportMatrix a document that explains for yourdevelopers and end users whats allowed inproduction DR QA and development Thishelps set expectations going forward - if aservers going to be production then it has

to be stable and that means making surechanges dont happen accidentally

I created that sample support matrix when I

worked as a DBA and Ive shared it so youcan do a save-as and get a fast start on your

own Hope that helps

Most DBAs donrsquot

actually have a list of

their servers

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638

copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds

by Brent Ozar

The only reason we dobackups is so we can dorestores

When I 1047297rst started outas a SQL Server DBA Ithought things were

going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent

every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be

In theory you test our backup strategyahead of time with Kendras 5 Simple

Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers

In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its

just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a

database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier

Where to Do Restores

When youre restoring code (storedprocedures views triggers etc) or

individual tables dont restore onto the

production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a

diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments

Afer weve safely restored the right dataonto another server its easy to copy that

data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked

server tables

However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the

data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase

This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that

Bootcamps and

certi 1047297 cation

programs alone wonrsquot get

you that 1047297 rst jobmdash not when

your competition has

experience

(And yes this means wersquore assuming yoursquore backing them up)

The Reason We Back Up

All Those Databases

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738

copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds

particular server didnt have any otheractivity happening due to the outage Were

not always so lucky but it helps to think outof the box like that

A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that

youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase

Doing the Restore

Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to

apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore

When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or

transaction logbackups The goal is to

1047297nish the restorequickly

Next if diff erential

backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one

Next restore all of the transaction log

backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -

again using WITH NORECOVERYDoing all of this with the GUI sucks The

more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and

restores them for you automatically in orderWell talk about automating restores in the

next training module

To learn more about backups and restoresour favorite geting-started articles are

bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA

bull Brents DBA Nightmare SQL Down NoPlans

bull Jess 3 Things You Need to Start Doing toYour Database Server

When I Did My First Restore

I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and

early to print out the list of labels for the highschool graduates weddings babies and so

on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld

One morning as part of my

data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the

WHERE clause so Ideleted all of the photoshoots ever

Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the

restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it

Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested

Either the backups are

tested or yoursquore about

to be tested

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838

copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds

by Brent Ozar

If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way

too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction

logs)

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0100trn

bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn

bull MyDatabase_20130718_0400trn

bull MyDatabase_20130718_0500trn

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

In that scenario I took my full backup at

midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in

the real world but

its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)

If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les

live and presto the script will restore all ofthe 1047297les for you automatically

This Script Has Implications For You

You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and

see what backups have been done

It assumes the same databasepath when you restore If yourethe kind of DBA who likes to

change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and

The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves

As we script out a restore wersquoll learn how we need to design backups

Doing Faster Database Restores

Knowing how yoursquoll

restore aff ects how

you back up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938

copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds

development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le

names at restore time but theyre not quiteas elegant when it comes to restoring fulls

diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully

automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly

load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact

You can automate your own 1047297re drill testing

Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a

series of Agent jobs to restore a diff erentservers backups every day and then run

DBCC jobs on them This way I know Imgeting really good backups

Your Homework for This Chapter

You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way

Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or

staging environment If you get errors leavecomments on that blog post describing the

error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly

in a GUI - youll just open a proven script andhit Execute Bam

Geting Even Fancier

You can use this technique to build smallercopies of your production databases For

example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring

the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down

The end result was a database that was over50 smaller

I then backed it up with compression and

lef

those backups in a folder that thedevelopers could access That made it easier

for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired

Free 30-Minute Videos to Learn More

Log Shipping Part 1 Jes explains how logshipping is really just doing your restores

ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs

Backup Planning Video Jeremiah explains 3

company scenarios and asks you to pick theright backup strategy for each

Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about

backups We talk database mirroring

clustering AlwaysOn Availability Groupsand more

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038

copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds

by Brent Ozar

Run our free sp_Blitztradestored procedure on one of

your production servers andpay particular atention tothe Security section of theresults It lists the logins

who have been granted thesysadmin or security admin roles

Dont think of them as logins

Think of them as people who can get you1047297red

These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop

Youre probably not get

ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in

Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is

unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room

How to Get Started Locking Things Down

Before you start removing peoples SArights be aware that there can be political

backlash In one shop the head developers

SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in

it and then dropped the database Nobody

knew because it was only around for 30-45minutes The problem could have been

avoided by communicating the securitychanges ahead of time and thats where we

need to startTake the sp_Blitztrade output to your manager -

just the security portions - and say

something like this

Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont

want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that

Before we try to lock things down 1047297nd out who has copies of keys

Security Knowing Who Has

Access to the Servers amp Data

Think of sysadmin

logins as people

who can get you 1047297 red

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138

copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds

Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say

Were starting to lock down the productionserver but I want to make sure you have all

the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change

When You Get Resistance

When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a

large stake in the database For example ifthe accounting database is on the server goto the CFO and say

Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page

You want to turn it into a business problemnot a technology problem and the CFO will

very much be on your side She cant aff ordto have her entire department go down just

because some developer didnt put aWHERE clause on a T-SQL statement

I Know This Chapter Isnt Fun

Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks

In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details

Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those

applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want

Manage lots of

servers Learn

How to Be a

Senior DBA

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 4: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 438

copyBrent Ozar Unlimitedreg 2015 Page 4 htpBrentOzarcomneeds

Letrsquos start by making an Excel spreadsheet of the servers wersquove got the

number of databases on each one and over the coming weeks wersquoll 1047297x them

Build a Server Inventory

by Brent Ozar

At your company walk intothe VP of Saless officeand ask them how manysalespeople they have

NO I mean dont actuallyDO that because hes

going to ask you why the sales app is so

slow But I mean imagine if you COULD walkinto his office and ask him that I bet hewould have an instant answer He wouldntwait for a single moment Or walk into theCEOs office and ask how many employeeshe has Or ask the CFO how much the annualbudget is

My point is that when youre in charge youneed to know exactly what youre in chargeof

Make a Spreadsheet InventoryLets start by making a spreadsheet Acrossthe top make columns for

bull SQL Server Version (2012 2008 2005)

bull Edition (Standard Enterprise Developer)

bull Environment (Production QAdevelopment disaster recovery)

bull Department (sales HR accounting ITmixed use)

bull Business Users Aff ected (list of people toemail when the server dies)

bull Application Names (internal or externalproduct names)

bull Plan B

That last column gets a litle tricky - itmeans if this server dies in a 1047297re whats our

Plan B Are we going to restore thedatabases from another server Will we failover to a log shipped copy Or will weupdate our resume and head out for an earlylunch As we go farther into the trainingwere going to get much more speci1047297c about

Plan B

Theres no wrong answers here - week 1 isabout understanding where were at todaynot where wed like to be Were never where

wed like to be (Me personally Id like to beat a poolside bar right now but noooo Im ina hotel room waiting for my girlfriend to1047297nish blow drying her hair If youve everwondered why I write so much you can thankher full head of hair)

If youd like to get ambitious add additionalcolumns for Core Count CPU Count andMemory The core and CPU counts will getyou a head start on licensing although Ihave to confess that were not going to cover

licensing as part of our training plan

What Well Do With This Spreadsheet

Right now you probably sleep well at night

thinking you know everything thatshappening in these servers Hoooweee haveI got bad news for you Over the next sixmonths were going to progressively add

more and more columns to this spreadsheetas we learn more about our environment

uncover problems and learn how to solvethem

For bonus points add a column for WhatScares Me Write a quick note about the one

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 538

copyBrent Ozar Unlimitedreg 2015 Page 5 htpBrentOzarcomneeds

thing that scares youmost about this serverMaybe its blockingproblems maybe its the

failing jobs maybe itscode you dont

understand Six monthsfrom now I bet youll be proud of how thiscolumn has changed

How to Survey Your Network for Servers

Put a row in the spreadsheet for every serveryou have - whether youre in charge of it ornot We want to start with a good inventoryof what we have and theres two good free

tools to do it

Microsof Assessment and Planning Toolkit -

its actually designed for licensingcompliance but it works great for building

server inventories It scans your networklooking for whatever programs you pick but

just con1047297ne it to SQL Servers only

Quest Discovery Wizard for SQL Server - itsa GUI tool that pings all the servers in your

network and tries to 1047297gure out if theyve gotSQL Server installed If youre in a smallshop where your account has admin

privileges in the domain you might 1047297nd a lotmore servers than you expected

We dont get paid forplugging these productsand were always on thelookout for similar

inventory-building toolsso if you know of a beter

one email it to us atHelpBrentOzarcom

PSST Ask About This Before Geting Hired

When you take a new job as a DBA the very1047297rst question you should ask the company isDo you have a list handy of all the SQLServers Ill be managing I dont have to seethe list - I understand if you have security

concerns - but I just want to know if that listexists

Most of the timeit wont

This question serves two purposes it tellsYOU if the company has their act togetherwhen it comes to documentation and it tellsTHEM that youre the right person tomanage their database servers If they donthave the list theyre going to want that list

right away Nows your chance to explainhow you would go about gathering thatinformation (armed with the info in this

email)Bonus points create a SQL Server SupportMatrix a document that explains for yourdevelopers and end users whats allowed inproduction DR QA and development Thishelps set expectations going forward - if aservers going to be production then it has

to be stable and that means making surechanges dont happen accidentally

I created that sample support matrix when I

worked as a DBA and Ive shared it so youcan do a save-as and get a fast start on your

own Hope that helps

Most DBAs donrsquot

actually have a list of

their servers

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638

copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds

by Brent Ozar

The only reason we dobackups is so we can dorestores

When I 1047297rst started outas a SQL Server DBA Ithought things were

going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent

every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be

In theory you test our backup strategyahead of time with Kendras 5 Simple

Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers

In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its

just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a

database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier

Where to Do Restores

When youre restoring code (storedprocedures views triggers etc) or

individual tables dont restore onto the

production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a

diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments

Afer weve safely restored the right dataonto another server its easy to copy that

data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked

server tables

However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the

data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase

This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that

Bootcamps and

certi 1047297 cation

programs alone wonrsquot get

you that 1047297 rst jobmdash not when

your competition has

experience

(And yes this means wersquore assuming yoursquore backing them up)

The Reason We Back Up

All Those Databases

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738

copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds

particular server didnt have any otheractivity happening due to the outage Were

not always so lucky but it helps to think outof the box like that

A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that

youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase

Doing the Restore

Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to

apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore

When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or

transaction logbackups The goal is to

1047297nish the restorequickly

Next if diff erential

backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one

Next restore all of the transaction log

backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -

again using WITH NORECOVERYDoing all of this with the GUI sucks The

more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and

restores them for you automatically in orderWell talk about automating restores in the

next training module

To learn more about backups and restoresour favorite geting-started articles are

bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA

bull Brents DBA Nightmare SQL Down NoPlans

bull Jess 3 Things You Need to Start Doing toYour Database Server

When I Did My First Restore

I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and

early to print out the list of labels for the highschool graduates weddings babies and so

on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld

One morning as part of my

data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the

WHERE clause so Ideleted all of the photoshoots ever

Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the

restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it

Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested

Either the backups are

tested or yoursquore about

to be tested

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838

copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds

by Brent Ozar

If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way

too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction

logs)

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0100trn

bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn

bull MyDatabase_20130718_0400trn

bull MyDatabase_20130718_0500trn

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

In that scenario I took my full backup at

midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in

the real world but

its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)

If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les

live and presto the script will restore all ofthe 1047297les for you automatically

This Script Has Implications For You

You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and

see what backups have been done

It assumes the same databasepath when you restore If yourethe kind of DBA who likes to

change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and

The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves

As we script out a restore wersquoll learn how we need to design backups

Doing Faster Database Restores

Knowing how yoursquoll

restore aff ects how

you back up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938

copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds

development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le

names at restore time but theyre not quiteas elegant when it comes to restoring fulls

diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully

automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly

load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact

You can automate your own 1047297re drill testing

Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a

series of Agent jobs to restore a diff erentservers backups every day and then run

DBCC jobs on them This way I know Imgeting really good backups

Your Homework for This Chapter

You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way

Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or

staging environment If you get errors leavecomments on that blog post describing the

error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly

in a GUI - youll just open a proven script andhit Execute Bam

Geting Even Fancier

You can use this technique to build smallercopies of your production databases For

example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring

the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down

The end result was a database that was over50 smaller

I then backed it up with compression and

lef

those backups in a folder that thedevelopers could access That made it easier

for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired

Free 30-Minute Videos to Learn More

Log Shipping Part 1 Jes explains how logshipping is really just doing your restores

ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs

Backup Planning Video Jeremiah explains 3

company scenarios and asks you to pick theright backup strategy for each

Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about

backups We talk database mirroring

clustering AlwaysOn Availability Groupsand more

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038

copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds

by Brent Ozar

Run our free sp_Blitztradestored procedure on one of

your production servers andpay particular atention tothe Security section of theresults It lists the logins

who have been granted thesysadmin or security admin roles

Dont think of them as logins

Think of them as people who can get you1047297red

These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop

Youre probably not get

ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in

Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is

unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room

How to Get Started Locking Things Down

Before you start removing peoples SArights be aware that there can be political

backlash In one shop the head developers

SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in

it and then dropped the database Nobody

knew because it was only around for 30-45minutes The problem could have been

avoided by communicating the securitychanges ahead of time and thats where we

need to startTake the sp_Blitztrade output to your manager -

just the security portions - and say

something like this

Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont

want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that

Before we try to lock things down 1047297nd out who has copies of keys

Security Knowing Who Has

Access to the Servers amp Data

Think of sysadmin

logins as people

who can get you 1047297 red

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138

copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds

Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say

Were starting to lock down the productionserver but I want to make sure you have all

the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change

When You Get Resistance

When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a

large stake in the database For example ifthe accounting database is on the server goto the CFO and say

Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page

You want to turn it into a business problemnot a technology problem and the CFO will

very much be on your side She cant aff ordto have her entire department go down just

because some developer didnt put aWHERE clause on a T-SQL statement

I Know This Chapter Isnt Fun

Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks

In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details

Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those

applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want

Manage lots of

servers Learn

How to Be a

Senior DBA

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 5: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 538

copyBrent Ozar Unlimitedreg 2015 Page 5 htpBrentOzarcomneeds

thing that scares youmost about this serverMaybe its blockingproblems maybe its the

failing jobs maybe itscode you dont

understand Six monthsfrom now I bet youll be proud of how thiscolumn has changed

How to Survey Your Network for Servers

Put a row in the spreadsheet for every serveryou have - whether youre in charge of it ornot We want to start with a good inventoryof what we have and theres two good free

tools to do it

Microsof Assessment and Planning Toolkit -

its actually designed for licensingcompliance but it works great for building

server inventories It scans your networklooking for whatever programs you pick but

just con1047297ne it to SQL Servers only

Quest Discovery Wizard for SQL Server - itsa GUI tool that pings all the servers in your

network and tries to 1047297gure out if theyve gotSQL Server installed If youre in a smallshop where your account has admin

privileges in the domain you might 1047297nd a lotmore servers than you expected

We dont get paid forplugging these productsand were always on thelookout for similar

inventory-building toolsso if you know of a beter

one email it to us atHelpBrentOzarcom

PSST Ask About This Before Geting Hired

When you take a new job as a DBA the very1047297rst question you should ask the company isDo you have a list handy of all the SQLServers Ill be managing I dont have to seethe list - I understand if you have security

concerns - but I just want to know if that listexists

Most of the timeit wont

This question serves two purposes it tellsYOU if the company has their act togetherwhen it comes to documentation and it tellsTHEM that youre the right person tomanage their database servers If they donthave the list theyre going to want that list

right away Nows your chance to explainhow you would go about gathering thatinformation (armed with the info in this

email)Bonus points create a SQL Server SupportMatrix a document that explains for yourdevelopers and end users whats allowed inproduction DR QA and development Thishelps set expectations going forward - if aservers going to be production then it has

to be stable and that means making surechanges dont happen accidentally

I created that sample support matrix when I

worked as a DBA and Ive shared it so youcan do a save-as and get a fast start on your

own Hope that helps

Most DBAs donrsquot

actually have a list of

their servers

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638

copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds

by Brent Ozar

The only reason we dobackups is so we can dorestores

When I 1047297rst started outas a SQL Server DBA Ithought things were

going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent

every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be

In theory you test our backup strategyahead of time with Kendras 5 Simple

Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers

In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its

just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a

database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier

Where to Do Restores

When youre restoring code (storedprocedures views triggers etc) or

individual tables dont restore onto the

production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a

diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments

Afer weve safely restored the right dataonto another server its easy to copy that

data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked

server tables

However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the

data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase

This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that

Bootcamps and

certi 1047297 cation

programs alone wonrsquot get

you that 1047297 rst jobmdash not when

your competition has

experience

(And yes this means wersquore assuming yoursquore backing them up)

The Reason We Back Up

All Those Databases

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738

copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds

particular server didnt have any otheractivity happening due to the outage Were

not always so lucky but it helps to think outof the box like that

A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that

youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase

Doing the Restore

Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to

apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore

When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or

transaction logbackups The goal is to

1047297nish the restorequickly

Next if diff erential

backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one

Next restore all of the transaction log

backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -

again using WITH NORECOVERYDoing all of this with the GUI sucks The

more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and

restores them for you automatically in orderWell talk about automating restores in the

next training module

To learn more about backups and restoresour favorite geting-started articles are

bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA

bull Brents DBA Nightmare SQL Down NoPlans

bull Jess 3 Things You Need to Start Doing toYour Database Server

When I Did My First Restore

I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and

early to print out the list of labels for the highschool graduates weddings babies and so

on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld

One morning as part of my

data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the

WHERE clause so Ideleted all of the photoshoots ever

Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the

restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it

Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested

Either the backups are

tested or yoursquore about

to be tested

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838

copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds

by Brent Ozar

If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way

too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction

logs)

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0100trn

bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn

bull MyDatabase_20130718_0400trn

bull MyDatabase_20130718_0500trn

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

In that scenario I took my full backup at

midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in

the real world but

its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)

If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les

live and presto the script will restore all ofthe 1047297les for you automatically

This Script Has Implications For You

You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and

see what backups have been done

It assumes the same databasepath when you restore If yourethe kind of DBA who likes to

change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and

The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves

As we script out a restore wersquoll learn how we need to design backups

Doing Faster Database Restores

Knowing how yoursquoll

restore aff ects how

you back up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938

copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds

development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le

names at restore time but theyre not quiteas elegant when it comes to restoring fulls

diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully

automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly

load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact

You can automate your own 1047297re drill testing

Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a

series of Agent jobs to restore a diff erentservers backups every day and then run

DBCC jobs on them This way I know Imgeting really good backups

Your Homework for This Chapter

You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way

Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or

staging environment If you get errors leavecomments on that blog post describing the

error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly

in a GUI - youll just open a proven script andhit Execute Bam

Geting Even Fancier

You can use this technique to build smallercopies of your production databases For

example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring

the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down

The end result was a database that was over50 smaller

I then backed it up with compression and

lef

those backups in a folder that thedevelopers could access That made it easier

for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired

Free 30-Minute Videos to Learn More

Log Shipping Part 1 Jes explains how logshipping is really just doing your restores

ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs

Backup Planning Video Jeremiah explains 3

company scenarios and asks you to pick theright backup strategy for each

Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about

backups We talk database mirroring

clustering AlwaysOn Availability Groupsand more

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038

copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds

by Brent Ozar

Run our free sp_Blitztradestored procedure on one of

your production servers andpay particular atention tothe Security section of theresults It lists the logins

who have been granted thesysadmin or security admin roles

Dont think of them as logins

Think of them as people who can get you1047297red

These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop

Youre probably not get

ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in

Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is

unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room

How to Get Started Locking Things Down

Before you start removing peoples SArights be aware that there can be political

backlash In one shop the head developers

SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in

it and then dropped the database Nobody

knew because it was only around for 30-45minutes The problem could have been

avoided by communicating the securitychanges ahead of time and thats where we

need to startTake the sp_Blitztrade output to your manager -

just the security portions - and say

something like this

Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont

want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that

Before we try to lock things down 1047297nd out who has copies of keys

Security Knowing Who Has

Access to the Servers amp Data

Think of sysadmin

logins as people

who can get you 1047297 red

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138

copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds

Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say

Were starting to lock down the productionserver but I want to make sure you have all

the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change

When You Get Resistance

When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a

large stake in the database For example ifthe accounting database is on the server goto the CFO and say

Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page

You want to turn it into a business problemnot a technology problem and the CFO will

very much be on your side She cant aff ordto have her entire department go down just

because some developer didnt put aWHERE clause on a T-SQL statement

I Know This Chapter Isnt Fun

Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks

In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details

Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those

applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want

Manage lots of

servers Learn

How to Be a

Senior DBA

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 6: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638

copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds

by Brent Ozar

The only reason we dobackups is so we can dorestores

When I 1047297rst started outas a SQL Server DBA Ithought things were

going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent

every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be

In theory you test our backup strategyahead of time with Kendras 5 Simple

Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers

In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its

just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a

database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier

Where to Do Restores

When youre restoring code (storedprocedures views triggers etc) or

individual tables dont restore onto the

production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a

diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments

Afer weve safely restored the right dataonto another server its easy to copy that

data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked

server tables

However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the

data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase

This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that

Bootcamps and

certi 1047297 cation

programs alone wonrsquot get

you that 1047297 rst jobmdash not when

your competition has

experience

(And yes this means wersquore assuming yoursquore backing them up)

The Reason We Back Up

All Those Databases

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738

copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds

particular server didnt have any otheractivity happening due to the outage Were

not always so lucky but it helps to think outof the box like that

A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that

youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase

Doing the Restore

Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to

apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore

When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or

transaction logbackups The goal is to

1047297nish the restorequickly

Next if diff erential

backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one

Next restore all of the transaction log

backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -

again using WITH NORECOVERYDoing all of this with the GUI sucks The

more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and

restores them for you automatically in orderWell talk about automating restores in the

next training module

To learn more about backups and restoresour favorite geting-started articles are

bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA

bull Brents DBA Nightmare SQL Down NoPlans

bull Jess 3 Things You Need to Start Doing toYour Database Server

When I Did My First Restore

I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and

early to print out the list of labels for the highschool graduates weddings babies and so

on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld

One morning as part of my

data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the

WHERE clause so Ideleted all of the photoshoots ever

Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the

restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it

Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested

Either the backups are

tested or yoursquore about

to be tested

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838

copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds

by Brent Ozar

If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way

too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction

logs)

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0100trn

bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn

bull MyDatabase_20130718_0400trn

bull MyDatabase_20130718_0500trn

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

In that scenario I took my full backup at

midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in

the real world but

its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)

If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les

live and presto the script will restore all ofthe 1047297les for you automatically

This Script Has Implications For You

You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and

see what backups have been done

It assumes the same databasepath when you restore If yourethe kind of DBA who likes to

change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and

The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves

As we script out a restore wersquoll learn how we need to design backups

Doing Faster Database Restores

Knowing how yoursquoll

restore aff ects how

you back up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938

copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds

development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le

names at restore time but theyre not quiteas elegant when it comes to restoring fulls

diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully

automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly

load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact

You can automate your own 1047297re drill testing

Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a

series of Agent jobs to restore a diff erentservers backups every day and then run

DBCC jobs on them This way I know Imgeting really good backups

Your Homework for This Chapter

You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way

Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or

staging environment If you get errors leavecomments on that blog post describing the

error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly

in a GUI - youll just open a proven script andhit Execute Bam

Geting Even Fancier

You can use this technique to build smallercopies of your production databases For

example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring

the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down

The end result was a database that was over50 smaller

I then backed it up with compression and

lef

those backups in a folder that thedevelopers could access That made it easier

for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired

Free 30-Minute Videos to Learn More

Log Shipping Part 1 Jes explains how logshipping is really just doing your restores

ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs

Backup Planning Video Jeremiah explains 3

company scenarios and asks you to pick theright backup strategy for each

Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about

backups We talk database mirroring

clustering AlwaysOn Availability Groupsand more

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038

copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds

by Brent Ozar

Run our free sp_Blitztradestored procedure on one of

your production servers andpay particular atention tothe Security section of theresults It lists the logins

who have been granted thesysadmin or security admin roles

Dont think of them as logins

Think of them as people who can get you1047297red

These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop

Youre probably not get

ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in

Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is

unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room

How to Get Started Locking Things Down

Before you start removing peoples SArights be aware that there can be political

backlash In one shop the head developers

SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in

it and then dropped the database Nobody

knew because it was only around for 30-45minutes The problem could have been

avoided by communicating the securitychanges ahead of time and thats where we

need to startTake the sp_Blitztrade output to your manager -

just the security portions - and say

something like this

Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont

want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that

Before we try to lock things down 1047297nd out who has copies of keys

Security Knowing Who Has

Access to the Servers amp Data

Think of sysadmin

logins as people

who can get you 1047297 red

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138

copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds

Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say

Were starting to lock down the productionserver but I want to make sure you have all

the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change

When You Get Resistance

When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a

large stake in the database For example ifthe accounting database is on the server goto the CFO and say

Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page

You want to turn it into a business problemnot a technology problem and the CFO will

very much be on your side She cant aff ordto have her entire department go down just

because some developer didnt put aWHERE clause on a T-SQL statement

I Know This Chapter Isnt Fun

Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks

In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details

Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those

applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want

Manage lots of

servers Learn

How to Be a

Senior DBA

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 7: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738

copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds

particular server didnt have any otheractivity happening due to the outage Were

not always so lucky but it helps to think outof the box like that

A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that

youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase

Doing the Restore

Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to

apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore

When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or

transaction logbackups The goal is to

1047297nish the restorequickly

Next if diff erential

backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one

Next restore all of the transaction log

backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -

again using WITH NORECOVERYDoing all of this with the GUI sucks The

more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and

restores them for you automatically in orderWell talk about automating restores in the

next training module

To learn more about backups and restoresour favorite geting-started articles are

bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA

bull Brents DBA Nightmare SQL Down NoPlans

bull Jess 3 Things You Need to Start Doing toYour Database Server

When I Did My First Restore

I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and

early to print out the list of labels for the highschool graduates weddings babies and so

on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld

One morning as part of my

data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the

WHERE clause so Ideleted all of the photoshoots ever

Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the

restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it

Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested

Either the backups are

tested or yoursquore about

to be tested

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838

copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds

by Brent Ozar

If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way

too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction

logs)

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0100trn

bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn

bull MyDatabase_20130718_0400trn

bull MyDatabase_20130718_0500trn

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

In that scenario I took my full backup at

midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in

the real world but

its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)

If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les

live and presto the script will restore all ofthe 1047297les for you automatically

This Script Has Implications For You

You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and

see what backups have been done

It assumes the same databasepath when you restore If yourethe kind of DBA who likes to

change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and

The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves

As we script out a restore wersquoll learn how we need to design backups

Doing Faster Database Restores

Knowing how yoursquoll

restore aff ects how

you back up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938

copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds

development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le

names at restore time but theyre not quiteas elegant when it comes to restoring fulls

diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully

automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly

load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact

You can automate your own 1047297re drill testing

Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a

series of Agent jobs to restore a diff erentservers backups every day and then run

DBCC jobs on them This way I know Imgeting really good backups

Your Homework for This Chapter

You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way

Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or

staging environment If you get errors leavecomments on that blog post describing the

error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly

in a GUI - youll just open a proven script andhit Execute Bam

Geting Even Fancier

You can use this technique to build smallercopies of your production databases For

example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring

the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down

The end result was a database that was over50 smaller

I then backed it up with compression and

lef

those backups in a folder that thedevelopers could access That made it easier

for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired

Free 30-Minute Videos to Learn More

Log Shipping Part 1 Jes explains how logshipping is really just doing your restores

ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs

Backup Planning Video Jeremiah explains 3

company scenarios and asks you to pick theright backup strategy for each

Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about

backups We talk database mirroring

clustering AlwaysOn Availability Groupsand more

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038

copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds

by Brent Ozar

Run our free sp_Blitztradestored procedure on one of

your production servers andpay particular atention tothe Security section of theresults It lists the logins

who have been granted thesysadmin or security admin roles

Dont think of them as logins

Think of them as people who can get you1047297red

These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop

Youre probably not get

ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in

Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is

unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room

How to Get Started Locking Things Down

Before you start removing peoples SArights be aware that there can be political

backlash In one shop the head developers

SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in

it and then dropped the database Nobody

knew because it was only around for 30-45minutes The problem could have been

avoided by communicating the securitychanges ahead of time and thats where we

need to startTake the sp_Blitztrade output to your manager -

just the security portions - and say

something like this

Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont

want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that

Before we try to lock things down 1047297nd out who has copies of keys

Security Knowing Who Has

Access to the Servers amp Data

Think of sysadmin

logins as people

who can get you 1047297 red

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138

copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds

Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say

Were starting to lock down the productionserver but I want to make sure you have all

the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change

When You Get Resistance

When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a

large stake in the database For example ifthe accounting database is on the server goto the CFO and say

Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page

You want to turn it into a business problemnot a technology problem and the CFO will

very much be on your side She cant aff ordto have her entire department go down just

because some developer didnt put aWHERE clause on a T-SQL statement

I Know This Chapter Isnt Fun

Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks

In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details

Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those

applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want

Manage lots of

servers Learn

How to Be a

Senior DBA

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 8: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838

copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds

by Brent Ozar

If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way

too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction

logs)

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0100trn

bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn

bull MyDatabase_20130718_0400trn

bull MyDatabase_20130718_0500trn

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

In that scenario I took my full backup at

midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in

the real world but

its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)

If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this

bull MyDatabase_20130718_0000bak

bull MyDatabase_20130718_0600dif

bull MyDatabase_20130718_0700trn

bull MyDatabase_20130718_0800trn

Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les

live and presto the script will restore all ofthe 1047297les for you automatically

This Script Has Implications For You

You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and

see what backups have been done

It assumes the same databasepath when you restore If yourethe kind of DBA who likes to

change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and

The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves

As we script out a restore wersquoll learn how we need to design backups

Doing Faster Database Restores

Knowing how yoursquoll

restore aff ects how

you back up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938

copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds

development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le

names at restore time but theyre not quiteas elegant when it comes to restoring fulls

diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully

automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly

load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact

You can automate your own 1047297re drill testing

Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a

series of Agent jobs to restore a diff erentservers backups every day and then run

DBCC jobs on them This way I know Imgeting really good backups

Your Homework for This Chapter

You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way

Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or

staging environment If you get errors leavecomments on that blog post describing the

error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly

in a GUI - youll just open a proven script andhit Execute Bam

Geting Even Fancier

You can use this technique to build smallercopies of your production databases For

example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring

the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down

The end result was a database that was over50 smaller

I then backed it up with compression and

lef

those backups in a folder that thedevelopers could access That made it easier

for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired

Free 30-Minute Videos to Learn More

Log Shipping Part 1 Jes explains how logshipping is really just doing your restores

ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs

Backup Planning Video Jeremiah explains 3

company scenarios and asks you to pick theright backup strategy for each

Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about

backups We talk database mirroring

clustering AlwaysOn Availability Groupsand more

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038

copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds

by Brent Ozar

Run our free sp_Blitztradestored procedure on one of

your production servers andpay particular atention tothe Security section of theresults It lists the logins

who have been granted thesysadmin or security admin roles

Dont think of them as logins

Think of them as people who can get you1047297red

These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop

Youre probably not get

ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in

Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is

unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room

How to Get Started Locking Things Down

Before you start removing peoples SArights be aware that there can be political

backlash In one shop the head developers

SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in

it and then dropped the database Nobody

knew because it was only around for 30-45minutes The problem could have been

avoided by communicating the securitychanges ahead of time and thats where we

need to startTake the sp_Blitztrade output to your manager -

just the security portions - and say

something like this

Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont

want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that

Before we try to lock things down 1047297nd out who has copies of keys

Security Knowing Who Has

Access to the Servers amp Data

Think of sysadmin

logins as people

who can get you 1047297 red

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138

copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds

Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say

Were starting to lock down the productionserver but I want to make sure you have all

the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change

When You Get Resistance

When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a

large stake in the database For example ifthe accounting database is on the server goto the CFO and say

Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page

You want to turn it into a business problemnot a technology problem and the CFO will

very much be on your side She cant aff ordto have her entire department go down just

because some developer didnt put aWHERE clause on a T-SQL statement

I Know This Chapter Isnt Fun

Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks

In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details

Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those

applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want

Manage lots of

servers Learn

How to Be a

Senior DBA

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 9: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938

copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds

development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le

names at restore time but theyre not quiteas elegant when it comes to restoring fulls

diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully

automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly

load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact

You can automate your own 1047297re drill testing

Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a

series of Agent jobs to restore a diff erentservers backups every day and then run

DBCC jobs on them This way I know Imgeting really good backups

Your Homework for This Chapter

You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way

Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or

staging environment If you get errors leavecomments on that blog post describing the

error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly

in a GUI - youll just open a proven script andhit Execute Bam

Geting Even Fancier

You can use this technique to build smallercopies of your production databases For

example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring

the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down

The end result was a database that was over50 smaller

I then backed it up with compression and

lef

those backups in a folder that thedevelopers could access That made it easier

for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired

Free 30-Minute Videos to Learn More

Log Shipping Part 1 Jes explains how logshipping is really just doing your restores

ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs

Backup Planning Video Jeremiah explains 3

company scenarios and asks you to pick theright backup strategy for each

Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about

backups We talk database mirroring

clustering AlwaysOn Availability Groupsand more

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038

copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds

by Brent Ozar

Run our free sp_Blitztradestored procedure on one of

your production servers andpay particular atention tothe Security section of theresults It lists the logins

who have been granted thesysadmin or security admin roles

Dont think of them as logins

Think of them as people who can get you1047297red

These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop

Youre probably not get

ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in

Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is

unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room

How to Get Started Locking Things Down

Before you start removing peoples SArights be aware that there can be political

backlash In one shop the head developers

SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in

it and then dropped the database Nobody

knew because it was only around for 30-45minutes The problem could have been

avoided by communicating the securitychanges ahead of time and thats where we

need to startTake the sp_Blitztrade output to your manager -

just the security portions - and say

something like this

Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont

want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that

Before we try to lock things down 1047297nd out who has copies of keys

Security Knowing Who Has

Access to the Servers amp Data

Think of sysadmin

logins as people

who can get you 1047297 red

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138

copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds

Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say

Were starting to lock down the productionserver but I want to make sure you have all

the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change

When You Get Resistance

When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a

large stake in the database For example ifthe accounting database is on the server goto the CFO and say

Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page

You want to turn it into a business problemnot a technology problem and the CFO will

very much be on your side She cant aff ordto have her entire department go down just

because some developer didnt put aWHERE clause on a T-SQL statement

I Know This Chapter Isnt Fun

Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks

In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details

Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those

applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want

Manage lots of

servers Learn

How to Be a

Senior DBA

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 10: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038

copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds

by Brent Ozar

Run our free sp_Blitztradestored procedure on one of

your production servers andpay particular atention tothe Security section of theresults It lists the logins

who have been granted thesysadmin or security admin roles

Dont think of them as logins

Think of them as people who can get you1047297red

These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop

Youre probably not get

ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in

Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is

unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room

How to Get Started Locking Things Down

Before you start removing peoples SArights be aware that there can be political

backlash In one shop the head developers

SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in

it and then dropped the database Nobody

knew because it was only around for 30-45minutes The problem could have been

avoided by communicating the securitychanges ahead of time and thats where we

need to startTake the sp_Blitztrade output to your manager -

just the security portions - and say

something like this

Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont

want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that

Before we try to lock things down 1047297nd out who has copies of keys

Security Knowing Who Has

Access to the Servers amp Data

Think of sysadmin

logins as people

who can get you 1047297 red

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138

copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds

Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say

Were starting to lock down the productionserver but I want to make sure you have all

the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change

When You Get Resistance

When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a

large stake in the database For example ifthe accounting database is on the server goto the CFO and say

Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page

You want to turn it into a business problemnot a technology problem and the CFO will

very much be on your side She cant aff ordto have her entire department go down just

because some developer didnt put aWHERE clause on a T-SQL statement

I Know This Chapter Isnt Fun

Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks

In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details

Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those

applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want

Manage lots of

servers Learn

How to Be a

Senior DBA

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 11: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138

copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds

Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say

Were starting to lock down the productionserver but I want to make sure you have all

the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change

When You Get Resistance

When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a

large stake in the database For example ifthe accounting database is on the server goto the CFO and say

Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page

You want to turn it into a business problemnot a technology problem and the CFO will

very much be on your side She cant aff ordto have her entire department go down just

because some developer didnt put aWHERE clause on a T-SQL statement

I Know This Chapter Isnt Fun

Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks

In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details

Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those

applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want

Manage lots of

servers Learn

How to Be a

Senior DBA

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 12: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238

copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds

to at least know the business purpose andthe business contact

The business contact is the one who really

helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it

youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance

too so now is a good time to start 1047298eshingout that part of the spreadsheet

Database Administration is Politics

Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want

Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and

sometimes its even diff erent companiesWe might be administering a database

server that houses a third-party applicationand the vendors support team demands tohave SA access

Consulting Lines

Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA

Sounds like youve got it all under control

What happens if that doesnt work

Would you mind driving

SQL Server needs a dog

High de1047297nition

video training on

your desktop

laptop or even

your iPad

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 13: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338

copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds

by Brent Ozar

Download the SQL ServerCompliance Guide Its an

old 92-page whitepapercirca 2008 but it might justbe the best (and most

timeless) technical document that Microsof

has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job

I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking

with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that

were actually doing the governance stuff wesay were doing

Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly

secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)

Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds

$ ()+-()+-$0+1$2+-

30) $2+--+ 4 567899lt=65

) $2+--+ 4 569gt865

) $2+--+ 4 56gtltA=B65

Feel free to change out those keywords for

other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it

If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the

news tomorrow

When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and

use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business

Compliance Knowing Who Accesses What

Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data

Top tip when you play Risk go for South America Itrsquos only 4 countries

with 2 access points so itrsquos easy to defend See security maters

Risk isnrsquot just a board game

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 14: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438

copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds

Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -

things like SQL Server Auditing In practicethough this is a problem for hard-core

security teamsbecause the verysame DBA whomanages the

servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable

auditing get the necessary data and then

enable it again

With seriously ambitious work you can lockauditing down so that cant happen but its really really hard

As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys

third party hardware appliances like IBMGuardium or Imperva that act like a network

1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without

you (the DBA) knowing about it

These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database

suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment

If you just want to check a box and make theauditors think youre secure thats easy and

cheap but seriously good security isseriously expensive

Big Companies Love Security

The bigger the company the more interestedthey get in security

In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and

everybody knowswhere all the

passwords are stored

In big shops with a dedicated security teamthe security team usually makes requests to

disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)

This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request

easier For more info check out the securityenhancements section of Whats New in2014

Learning More About Security

Check out Denny Cherrys book Securing

SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life

Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a

copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement

Know whether your

company just wants

to check a box on an auditorrsquos

form or really be secure

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 15: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538

copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds

by Brent Ozar

Wersquove coveredbackups and securityexcept

Well we havenrsquot

We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored

procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda

yadda yaddaOur point of this ebook is a fast start

Your journey in SQL Server databaseadministration is never really done Yoursquore

going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough

to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would

have gone deeper into clustering setupsreplication internals or whatever other

Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook

When you 1047297nd an area that you want to digdeeper into check out

Our video training courses - from our $29

90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes

Our in-person classes - we have 2-3 daycourses for developers and DBAs who need

to make SQL Server faster and morereliable

Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and

train you on the 1047297xes

In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two

layers backups and security Before we move up a level letrsquos pause

Itrsquos Time to Level Up

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 16: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638

copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds

by Brent Ozar

Donrsquot

Itrsquos not your job

No seriously hang on asecond Im not saying that

you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space

This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter

(You might actually even BE a Windows

admin just here because you need to learn

about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)

In our journey from the base of our Hierarchy

of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like

xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the

drive space monitoring to the pros

Why You Shouldnt Build a Monitoring Tool

If you want to be a professional developeryou should build things You should learn

what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a

killer living for yourself

But if you want to be a professional DBAyou should leave the developing to thedevelopers

Im not saying you shouldnt learn thedynamic management views (DMVs) how to

dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six

months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down

security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers

Building a crappy monitoring tool in your

spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring

Just when you thought it was safe to let your guard down

we throw a trick chapter at you

How to Monitor Drive Space

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 17: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738

copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds

tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)

How to Pick a Monitoring Tool

Theres basically three kinds of monitoring

tools out there for SQL Server

Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use

HP System Management Homepage

Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the

server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor

Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help

performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor

QuestFoglightPerformanceAnalysis and

to someextent the

tools in theabovecategory too

If I was you Id start by asking the Windowsteam if theyve got any sofware that handles

the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being

sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category

Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance

Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused

your phone to ring afer hours Heres someof my personal favorites

bull Deadlocks

bull Out-of-control query running for hours

bull Long-running job like slow backup

bull Queries that desperately need an index

bull SQL Server not accepting connections

Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a

couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the

problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause

quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by

get

ing you right to thesolution

Afer youve done yourevaluation and picked a

favorite get a quote from them - and get a

quote from the other vendors as well Sharethe quotes with the competing salespeople

If yoursquore building a

monitoring tool in your

spare time at work itrsquos either

going to be crappy or yoursquore

about to be laid off because you

have too much spare time

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 18: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838

copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds

Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want

Your Homework

I dont expect you to magically get budget

approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours

troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road

Ill also link you to a video showing you howto get the budget approval for that tool

Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive

any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs

How This Philosophy Cost Me a Job

I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to

monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework

Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer

is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself

If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor

Bobcats per 100 Orders andOther Spurious Metrics

by Jeremiah Peschka

Did you know that you canship a bobcat 130th of the

time and still maintain 97positive feedback on ebay

What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why

Monitoring SSDPerformance

by Jeremiah Peschka

What if we could watch SSDwear in real time Manyvendors off er SMART status

codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post

Before you build

or buy check out

the best free

SQL Server

downloads

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 19: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938

copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds

While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel

I very much want you to know whatrsquos going on inside each database

Manage Space Inside Databases

by Brent Ozar

Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages

Thats kilobytes - notmegabytes not gigabytes

but just mere kilobytes

Say that we create a table$) C =D1E7FAA9

GE7FAAH - H-IGJKJLK

E7FAA-8EA M)$0)GNOOLL

First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to

keep this email short Now see what you did

Its long again Doggone it its hard teachingthis stuff in an email

Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an

INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each

character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a

character) so we need 10 bytes to store it

If all of our employees average about 10characters in their name that means we

could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to

use some parts of the page to storemetadata and well talk about that later in

the training)

Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB

page off disk brings it into memory makesthe necessary changes and then writes that

data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows

records - and each page belongs exclusivelyto just one object

A Word About Objects

Youll notice that I avoid using the word

table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these

three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order

Clustered Index - what we normally think ofas a table If Id have created my table likethis

$) C =D1E7FAA9

GE7FAAH - H-IGJKJL P)+)I4I $2)HK

E7FAA-8EA M)$0)GNOOLL

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 20: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038

copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds

Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning

through all of my employees The clusteredindex is sorted in the order of the

EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName

Nonclustered index - If I tell SQL Server to

$) --$2)H -HQ Q-8EA

- =D1E7FAA9GE7FAA-8EAL

Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will

only include the1047297elds speci1047297ed in my

index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my

clustering keyEmployeeID)

All three of these objects - heaps clustered

indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index

basics read Jes Borlands SQL Server IndexTerms

Pages amp Rows on Disk

The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all

of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with

a small table because some of theparameters for this DMF will cause SQL

Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can

slow down your running SQL Server

This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a

VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure

just yet - you can easilybreak applications whendatatypes change Letsleave that for later

You can get similar metadata much faster byusing sp_spaceused but it doesnt get the

cool info about average record size and Iwanted to encourage you to go spelunkinghere

Learning More About Pages

In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom

database for demos and you actually getPDFs to print out and follow along as we go

Your Homework

Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database

Armed with thisknowledge of

pages yoursquoll make bet er

data modeling decisions

too (MAX) ainrsquot free

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 21: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138

copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds

from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The

details columns in the results will show howbig each index is and whether its geting

used or notThis is important because the more indexesyou have

bull The longer your backups take

bull The longer index rebuilds take

bull The more memory you need to cacheeverything

And most importantly the slower yourinsertsupdatesdeletes go because SQL

Server has to maintain more copies of yourtable

As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items

are so important

Want to See What a Page Looks Like

Check out the DBCC PAGE command Youpass in a database name 1047297le number and

page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood

This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB

page is helps you understand the output of

various Dynamic Management Views(DMVs) when they report back units inpages

Wersquove got tons of

blog posts and

videos about our

favorite indextips tricks and

free tools

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 22: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238

copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds

If everythingrsquos stored on an 8KB page and each page has only one object on

it then we can 1047297gure out what objects (tablesindexes) are geting cached

What Pages are In Memory

by Brent Ozar

We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL

Server uses memory forlots of things

bull Caching database objects

bull Sorting data for your query results

bull Caching execution plans

bull Performing system tasks

Ofen were surprised by how litle data isbeing cached for each database

Last section we looked at the 8KB pages inour database Those pages are the same

whether theyre on disk or in memory - theyinclude the database ID and the object ID so

if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be

aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more

$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K

$ =8B8D89A= 30- VNWXW 0-

5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA

()+ 9F91=E9DYZZAlt=A9gtlt7Blt9

[)2P CI HC-+G=8B8D89A=L K

=8B8D89A=

)H) CI J H$

Compare the size of each database versushow much is being cached Ofen in the 1047297eld

Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk

This Leads to Cool Questions

This DMV query leads to so many coolperformance tuning questions I get so

excited by these concepts

How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make

room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial

Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 23: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338

copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds

a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then

Are we caching low-value data If you mixvendor apps and in-house-writen apps on

the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most

important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the

memory on a SQL Server that needs to serveother applications

Do we have enough memory If yourerunning SQL Server 2008R212 Standard

Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a

VM or running Enterprise Edition thememory question gets a lot tougher To learn

more read A Sysadmins Guide to SQLServer Memory

Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your

SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead

Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively

querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A

nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want

The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk

When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help

Itrsquos Probably Not a SAN Problem

When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN

administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries

The SAN admin kept saying Its okay The

SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it

between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes

of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen

SAN caching is still great for writesespecially for the transaction log but dont

count on it helping for SELECT speeds

To learn more check out our training courseon Storage Virtualization and Hardware forDBAs

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 24: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438

copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds

by Brent Ozar

For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We

need to track

bull Phone number (required)

bull Billing contact last name (required)

bull Billing contact 1047297rst name (required)

bull Business name (optional)

bull Business category (restaurant doggroomer auto dealer etc)

bull Address 1

bull Address 2

bull City

bull State

bull Zip

bull Service start date

(Sometimes a person or a business will havemultiple phone numbers but for the sake of

this training lets keep it a simple 1047298at table)We will never have two records in here with

the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key

When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers

That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami

Throw in businesses and lets say our tablehas 500000 records in it

That means by default every time we insertone eensy litle record SQL Server has to

read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates

records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called

clustered because - well I have noidea why its called clustered but

the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number

Now we have the table organized by phonenumber and if we want to 1047297nd people by

phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our

customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in

In which we pretend the phone company but instead of giving everybody

unlimited calling we just organize the data

Indexes What Goes First

No Brentrsquoshome number

is not in this chapter

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 25: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538

copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds

The White Pages A Non-Clustered Index

Our customers constantly need to 1047297nd

peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would

create an index called the White Pagesbull Billing contact last name

bull Billing contact 1047297rst name

bull Phone number

That index would save people a ton of timeThink about how you use the white pages

1 You scan through pages looking at justthe leters at the top until you get close

2 When you get close you open up the fullbook and jump to the right leters

3 You can quickly 1047297nd the right single one

record

Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan

through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this

$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5

That doesnt say select the top one - it says

select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths

Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all

500000 records no mater what

If the database has an index by last name

and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith

Johnathan it knows it can stop becausetheres no more John Smiths

Covering FieldsHelping Indexes Help You

But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith

to call Af

er all if your name was JohnSmith and the phone book didnt include

your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too

bull Billing contact last name

bull Billing contact 1047297rst name

bull Address 1

bull Phone number

Do we absolutely need the address in ourindex for every query No but we include it

for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much

This is called a covering index because itcovers other 1047297elds that are useful Adding

the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we

need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 26: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638

copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds

skip When building covering indexes thecovering 1047297elds go at the end of the index

Obviously this index would suck

bull Billing contact last name

bull Address 1

bull Billing contact 1047297rst name

bull Phone number

We dont want all of the Smiths ordered bytheir address and then a jumbled mess of

1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those

Selectivity Why the Last Name Goes First

If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by

1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last

name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues

For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that

youre going to use in an index generally youput the most selective 1047297eld 1047297rst

Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -

but no mater how much of a suave guy youthink you are you dont really need ALL of

the women in Miami This is why non-selective indexes arent all that useful onlookup tables

This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 27: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738

copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds

The Yellow Pages Another Index

When we need to 1047297nd a dog groomer wedont want to go shuffling through the white

pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category

bull Business Category

bull Business Name

bull Address 1

bull Phone Number

Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the

records Here were searching for a range ofrecords not just a single one

Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business

Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several

records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index

Learning More About Indexes

Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes

Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more

Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs

SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals

Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one

Also available on Kindle

And 1047297nally if yoursquod like a video training

session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos

How to Tune

Indexes and

Speed Up SQL

6-Hour Training

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 28: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838

copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds

by Brent Ozar

Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to

spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me

Covering Indexes

Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query

$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5

And if I have this index

$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H

G(lt9B-8EAK PUA-YEDAltL

Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt

have to go back to the clustered index in

order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use

Covering indexes are most eff ective whenyou have very frequent queries that

constantly read data and theyre causingblocking problems or heavy IO

Filtered Indexes

Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place

orders We need to query orders that haventbeen processed yet like this

$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O

The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for

OrderProcessed = 1 Starting with SQL

Server 2008 we can create an index with aWHERE clause

$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)

lt=AltPltgtA99A= ] O

Covering1047297ltered full text XML heaps 3x5 index cards you name it

More Kinds of Indexes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 29: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938

copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds

Now we have an index on just a few records -

a small subset of our table

Full Text Indexes

Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says

$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965

But that wouldnt be veryefficient SQL Serverwould have to look at

every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row

Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c

words - but only as long as you rewrite yourquery to use the full text search commandslike this

$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L

You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out

bull Books Online on Full Text Indexing -seriously stop laughing the manuals

really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012

bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus

the diff erences between 2005 and 2008(which were huge)

XML Indexes

You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that

SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data

When you run XML queries SQL Server has

to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-

processed versions of

the XML so we can rapidly jump to speci1047297cnodes or values

bull Primary and Secondary XML Indexes -

Books Onlinebull Selective XML Indexes - instead of wasting

a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML

Heaps

Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts

When you want to query a heap SQL Server

scans the whole freakin thing Every SingleTime

Sounds bad right Most of the time it is -except for a couple of very niche uses If you

have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a

Why did it have

to be snakes

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 30: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038

copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds

staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test

it to make sure its actually faster under yourapplications needs

Picking the Right Indexes for Your Apps

SQL Servers Dynamic Management Views

(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw

data that has to be manually combined andinterpreted Well talk about that in coming

lessons but you need to have this groundknowledge of index options 1047297rst

Dude Who Stole My

Missing Index

Recommendation

by Kendra Litle

Recently Jes asked the team

an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo

I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent

index will start to fail (Thatrsquos a really badday)

Even so some people love index hints

Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation

How to Master Index

Tuning in One Step

by Kendra Litle

Irsquom going to tell you a secret

Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly

Herersquos that one step stop thinking indextuning is a problem for Future You

Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a

problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that

Index tuning isnrsquot something you do once a

year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will

alsochange As you tune indexes your query

plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month

I hear a lot of reasons why people donrsquot tunetheir indexes

Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server

index-tuning

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 31: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138

copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds

by Kendra Litle

Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated

frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey

Things went bad

Application performance plummeted Lots of

code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns

A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager

said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production

The DBA explained that fragmentation

wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented

Bad meet ugly

The whole performance team 1047298ipped out

Trust disappeared Managers squirmedMore managers were called in

The DBA tried to change the subject but it

was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing

Heres the deal-- the DBA was actually right

Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards

Why you need automated indexmaintenance

When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why

Regular index maintenance still has a lot of

merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting

loads of memory

Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it

helps more than it hurts

Indexes are like cars You have to maintain them yoursquore probably not doing

it and if you take them to a mechanic yoursquoll probably get overcharged

The Parable of Index Maintenance

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 32: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238

copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds

How much downtime canyou spare

Before you implement

index maintenance 1047297ndout how much time tablescan be offline in each of

your databases

If youve got SQL Server

Standard Edition indexrebuilds are alwaysoffline

Even with SQL Server

Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This

restriction is relaxed somewhat in SQLServer 2012)

Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014

Maintenance plans or custom scripts

You can go the easy way and use SQL ServerMaintenance Plans but unfortunately

theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can

be a decent option

If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance

scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance

Download and con1047297gure them on a test

instance 1047297rst Theres a lot of options on

parameters and youllneed to play with them

Get used to the cmdexec

job step types When youinstall the scripts youllsee that the SQL Server

Agent jobs run indexmaintenance using a callto sqlcmdexe in an

MSDOS style step Thatsby design

Use the examples on thewebsite If you scroll tothe botom of the index

maintenance page youll1047297nd all sorts of examples showing

how to get the procedure to do diff erentuseful things

Find out when maintenance fails

Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail

Tell your boss you did a good thing

Finally write up a quick summary of what you

did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep

Having your manager know youre taking the

time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out

Learning More About Fragmentation

5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that

Stop Worrying About Fragmentation -

defragging everything can cause moreproblems than it solves

Our Best Free

SQL Downloads

includes videotutorial on Olarsquos

script setup

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 33: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338

copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds

Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo

Backup amp Recovery Questions

1 How many production SQL Servers doyou have

2 Whats the RPO and RTO of your mostimportant production server

3 Did your backups take the normalamount of time last night

4 When was the last time DBCCsuccessfully 1047297nished in production

Security Questions

1 How many diff erent people aresysadmins in production

2 Do they each know that theyresysadmins and take care to avoidaccidents

3 How many of your databases hold

personally identi1047297able data like creditcard numbers social security numbersand passwords

4 If someone gets hold of one of those

database backups can your companysdata go public

5 Have you informed your managers ofthat risk or will they blame you

Monitoring Questions

1 When a database server runs out of drive

space who gets emailed

2 Do at least two diff erent people get theemail in case one is on vacation orunavailable

3 What actions will you take to1047297x thesituation

4 Are those actions documented so thateveryone who gets the email can take

action quickly

Index Questions

1 Does every table in production have aclustered index

2 For any exceptions (heaps) do you havea plan to 1047297x them

3 Which table in production has the most

indexes and why

4 Which frequently queried tables inproduction have the least indexes andwhy

5 How are you managing indexfragmentation

Pop Quiz

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 34: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438

copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds

The Right Answers

Theres no one right answer forany of these questions but

some answers are more wrongthan others Database administration is a

journey and not everyone in the company is

going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater

how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here

Youre not aiming for perfect

Youre aiming for good enough that yourmanagers accept the base of your Database

Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels

like performance and future-proo1047297ng

Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is

over Print out this email cut out thequestion list and scribble in a few thoughts

Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list

again Refresh your memory with the links onthe right side of this email

When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before

they start to help they have to make sureyour data is backed up and checked for

corruption They cant go making changeswithout having a safety net

And you shouldnt either

Remember that as we start talking aboutchanging server and database setings next

Donrsquot make changes

without a tested safety net

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 35: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538

copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds

As SQL Server runs queries it constantly tracks what it waits on

Ask SQL Server for these wait statistics and tuning is easy Easier anyway

What is SQL Server Waiting On

by Brent Ozar

You probably got intodatabase administrationby way of development orsystems administration

Youre used to monitoringstuff from the OUTSIDE

using things like Performance Monitorcounters

SQL Server has a way way way beter tool

When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve

up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time

Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and

goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on

The Crappy Way to Check Waits

Run this simple query

$ ()+ 9F91=E98B9B8B9

And youll get back a list of wait types plushow many milliseconds the server has spent

waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)

Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out

The Beter Way to Check Waits

Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the

1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds

If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something

If your server isnt busy your waits willprobably add up to 30 seconds - or much

You might be surprised at

how li t le memory is

used for caching data

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 36: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638

copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds

less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling

when the servers really busy

To learn more about wait types and what

they mean check out our wait typesresources page

Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already

Correlating Waits Stats and Perfmon

Once we think weve got a botleneck we

need to double-check those numbers bygathering server-level metrics about that

particular botleneck

In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats

youre seeing as a botleneck heres thePerfmon counters to collect

To double-check CXPACKET and

SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)

This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it

means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits

For PAGEIOLATCH and WRITELOG waits

Physical Disk Avg SecRead

Physical Disk Avg SecWrite

Physical Disk Avg ReadsSec

Physical Disk Avg WritesSec

The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage

Much like you the more work youre asked todo the slower you get The top two are the

SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing

You need to 1047297gure

out how to help an

ailing SQL Server

Our SQL Critical

Caretrade helps

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 37: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738

copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds

For LCK_ waits collect

SQLServer Locks - Lock Waitssec

SQL Server Locks - Avg Wait Time

SQL Server Access Methods - Table LockEscalationssec

SQL Server Transactions - Longest RunningTransaction Time

SQL Server Access Methods - Full Scanssec

These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a

lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of

other users

Got Other Waits

Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not

documented well To learn more about waittypes and what they mean check out ourwait types resources page

Learning More About Perfmon Counters

No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our

favorite resources

Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by

typeOur Perfmon tutorial - explaining how tocollect the data and analyze it

Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters

and thresholds

Watch Brent explain wait stats while hersquos

dressed up as Richard Simmons Click here

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us

Page 38: eBook SQL Server DBA Training Plan

8182019 eBook SQL Server DBA Training Plan

httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838

by Brent Ozar Unlimitedreg

We love - no we LOVE - helpingpeople get relief for data pains

and wersquove got lots of options tohelp

FIRST AID TOTALLY FREE STUFF

We build cool troubleshooting tools and givethem away for free

bull sp_Blitzreg - fast SQL Server health check

bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server

bull Our blog - thousands of articles onperformance tuning availability andcareer development

bull Much more - like our posters YouTubevideos and weekly webcasts

IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES

Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments

Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges

Join us in-person at our classes

VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW

bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain

bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more

bull Virtualization SANs and Hardware for

SQL Server $299 - 5 hours of subsystemsecrets

bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more

You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad

for 18 months

Thatrsquos just a sample - check out the full list

SQL CRITICAL CAREreg

A FASTER SAFER SERVER IN 4 DAYS

Tired of struggling with a slow unreliableSQL Server

In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop

We donrsquot keep secrets you get to keep our

scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment

Schedule a free 30-minute consultation with

us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom

Wersquove got more tricks than a pony

Get More Help from Us